Repository navigation
SCC26: Read WENO stencils directly from v_vf instead of an identical copy - #2092
AppleChen17 wants to merge 1 commit into
Conversation
The WENO3/5/7 reconstruction kernels read their stencil from v_rs_weno, which s_pack_weno_input_arr fills with an element-wise copy of v_vf using the same (j, k, l) layout. Read v_vf directly instead, which removes one read and one write of every reconstructed field per direction and per RK stage. v_rs_weno and the pack are still needed by MP-WENO (s_preserve_monotonicity) and THINC (s_thinc_compression), so they are allocated, filled and deallocated only when mp_weno or int_comp > 0. WENO1 and MUSCL do not use s_pack_weno_input_arr and are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lines of Code
|
|
Claude Code Review Head SHA: 4fd0f9e Files changed:
Findings: |
|
The two failing jobs, The failure happens before The Intel jobs install All other jobs pass, including the GNU jobs on Ubuntu and macOS and the NVHPC 25.5–26.3 CPU and GPU jobs. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #2092 +/- ##
==========================================
- Coverage 61.90% 61.89% -0.01%
==========================================
Files 86 86
Lines 22885 22888 +3
Branches 3369 3370 +1
==========================================
+ Hits 14166 14167 +1
- Misses 6220 6221 +1
- Partials 2499 2500 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
@AppleChen17 A maintainer will mark this as ready for review when it is suitable. Draft pull requests don't run on self hosted runners, which is preferable when they aren't ready yet. |
|
Thanks for the note, and sorry for the extra runner load. I've converted it back to draft and will leave the ready-for-review decision to the maintainers. |
Closes #2091.
Summary
For WENO3/5/7,
s_pack_weno_input_arrcopiesv_vfintov_rs_wenowith the same(j, k, l)layout before each reconstruction. This PR makes the WENO3/5/7 reconstruction kernels readv_vf(i)%sfdirectly.v_rs_wenois kept only for MP-WENO and THINC, which still take it as an argument. Background and the trade-off analysis are in #2091.On one H200, grind time drops 5.9–6.5% on three example cases with NVHPC 26.3 and 25.9. Outputs are bit-identical to
masteron our tested cases, and the full GPU and CPU test suites pass.This PR is from Team NTHU SCC, participating in the SC26 Student Cluster Competition.
Changes
Only
src/simulation/m_weno.fppchanges:v_rs_weno(${SF(...)}$, i)becomesv_vf(i)%sf(${SF(...)}$). The arithmetic and the loop structure are unchanged.v_rs_wenois allocated, packed and deallocated only whenmp_weno .or. int_comp > 0, fors_preserve_monotonicityands_thinc_compression.s_weno) are unchanged.Performance
Grind time (ns/gp/eq/rhs),
master→ this PR. Each entry is the mean of three interleaved runs within one allocation. All runs use H200, double precision,--case-optimizationand the default build flags. Two-GPU runs use two MPI ranks on one node.3D_performance_test(201³, WENO5, 30 steps)3D_shockdroplet(201×101×101, WENO3, 1000 steps)2D_shockdroplet --res 4(4800×1200, WENO3, 565 steps)Maximum run-to-run spread was 0.81% (NVHPC 26.3, 1 GPU), 0.61% (NVHPC 25.9, 1 GPU), and 2.21% for the two reported 2-GPU cases.
*The 2-GPU
3D_performance_testis omitted: on our node its run-to-run spread (up to 7.8%) was comparable to the effect.Where the time goes (NVHPC 26.3, Nsight Systems and Nsight Compute):
3D_performance_testand2D_shockdroplet. Removing it reduces total kernel time by 6.1% and 6.5%, respectively.master. See Avoid redundant WENO input packing in the common reconstruction path #2091 for details.Trade-off for MP-WENO / THINC
With
mp_wenoorint_comp > 0the copy is still needed, so the small reconstruction overhead is not offset. On2D_richtmyer_meshkov(WENO5 + MP-WENO, 431 steps), total kernel time is +0.8%, and grind time is +0.45%, within run-to-run variation (up to 2.3%). Avoiding this overhead would require separate reconstruction kernels selected at runtime. We chose not to introduce this duplication for the small observed regression.Unchanged paths behave as expected: total kernel time is +0.06% for
2D_orszag_tang(WENO1) and +0.03% for2D_shockdroplet_muscl(MUSCL + THINC), both with bit-identical outputs.Testing
Baseline:
masterat42c44fd.mp_weno=Ttests and the five WENO5int_comptests, which exercise the remaining copy path.master, including case-optimized builds. No golden files changed.CPU builds also use the new reads. CPU performance was not benchmarked.
Contribution Policy
All contributions are expected to demonstrate:
If these expectations are not met, we would prefer to implement the changes ourselves rather than spend time reviewing low-effort submissions.